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mvm^omm 

Wr**7 r -f7*7 !? -*«S#afcs SuIB^^VT 7 ^- 

su fufa^^vzx-^^^s^p/T^-r^^-f 

?B^[Wl^t:> hfl|«t LTBIJBfc:> h1fS8A7l#atf4 
HJBfcBMflfc V hflr«Jc*-pTIII«STaRllHilIS#«* 

su xhy-z*x-**j8#afrs»sft*xhy- 

[■HUH 3] yy-rT'WScDHtcBIWIWBSfiaafcdft 

mts xMj-A^-*£/##a^e*#6ftsxhu- 



[■98H6] B*3M5E«©^;Uf : ^7 r -f7 r BWIB»* 

iRlSBSiaffltV h1t«»c<fcoT«BB"r«RI»IIMl5*«* 
*§U tuEX h , J-/AiI^bi^7-^^<7(c 
J=oT5»S*h/fcBWk> h1»«tC«^T*5V7 7 W£ 

fift3fe»0)t > h*4A*B»k V MHBA»*a* 
S Ls t5IB^T t V7 7 x-^^^ISrj^W-r^^Y A^ 

u-/*^-££/$#atfx h 'j-Ax-^^fig-r^fs 

1tW*IBWk V h1t$B<h LTBIWb: V hit S8A*#atf 
MSW8] II^J17Ei8^^b^p<7^<7 7 (W]^i|gi5^ 

■leaser* *T*Tm^ftt>*firso?tT^7m3i^ 
A x - **im*T<<T wa#a-wiST? * « x - * tc 

JM8T3X h U-AIWH^at, B^Pfco^7 ; V7 7 P^ 

su xhu-A^-^^^a^etten^x hu- 

*6tifclRlfflb:v Ht«Hf\ ^n^nco/xV T'^BSE 
LTHuEBffiPS5^a^y7 r ^7 7 B^aiCfctt^^^ 

$ ^ -< a ^ >r > x v zl - y u l t m ? z> rwj m t 
vhwa^s^a^su xhu-A^-^^coKtc 

7 7 mi»IISB53S«o 

m&mi o] W5R3a9Ett^;u7 c y7 r -f7 , rafflinHs 
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Et- * * -f tws^imp en* * t 7 * <d 

HOBl»3S:ra»3t> hit 5BtC^oTiia5-r^lU^flii5^ 

W£ora»fMy*frft 3 #»*«*fc C 1 61WitS 
[000 1] 

[0 0 0 2] 

[0003] a^fi. H&wc^en^-SKftofcW 

tff^en^o COcfcdftSHTei*. CD-ROMfr5 

[0 0 0 4] «IBWW©3&afe»*-<r/e^ flUAtf, 
ICC D- R 0M±tcM^6n/c7 : -^CD7 7 KUX^^P 
^Alc J: y »sr C <!: TflWc LKM»J»*ff ft 3 

7WcCD-ROMiao TRS»'J Wft HUBS £*?5 



fffttotiSo 

[0 0 0 5] C<0cfc3ftBMI!B±*K3Rr*rc»Jc. c 
D - R O M&DEfl^lCttKO/ T 7 *— DflDx- 
* <fc LT-f V £ - <J - 7 LT$mZ-&Z> X h 'J -JUx- 

[0 0 0 6] 

[»«3^l»aLJ:«5trSilim L#LftffB±Ba£* 
X, Btf##frftto*U |Wl^tC^cDp<7 i V7 7 ^^^tl 

<ft*Ctti«*orco 

[0 0 0 7] c©ct3ft«BeiBWs:H2 5*«L^TK 

<7t=r-^ w tea l te * < ^ ^ > ht** y , 

«&<«&y* HEIOBOctdftttJBTtt, 7V*Ju 
£ UfaRffll*^ L<Z>3Ifttf»lcft 3SS LlSinicft o 

[0008] ca>cfc3fttt3i«:(B5»r*fc»^ cd- 

R O M«C0BB»f8tttcfel*£ x- £ OMtf *x*-r *ft 
^tc^^xV7 7 StcJ43iLT$Wi!r+L^CDT\ ^ft[B] 

[0 0 0 9] ±IBB3HlceftL, ^^xV 

[0 0 10] 

tl ^ xV T 7 x - £ £ £ -f Ix =? << VW«^ 6 X h y - A 

^stuiBx h y-ZxT^-^fes^atfx k 'j-Af-^ 
^ ftp^n^tc^ft 3 fc tixommty hitis^^ 
[ooi i] m&^2o^)\s**T*7mmmmtti 
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I*, m&m 1 Mmo^iit** ?<< ymmmmismtm 
tiraicajisu «HB>7 s -c7»w*#aH:c<o*-rz»5-f 

[0 0 12] ff^IM3CD-7/U^p<x<7 7 ^^P6i5 
tts P<7 1 V7'BSORgtcp)^illS5^T'3:-5rc4i)(7?|5l^t 

[0 0 13] H^4©^;l/^^7 :? 'r7 7 PR3ilili5^ 
tt» ff^JI3IBi2cDv;l/^p<X'r7 7 lll^illi5^«IE 

7. HJ-Z»8?fl!#l$£, iiufB><x-i'7'lS£#K©;><xV 
ZHIwraaB^PlSHtV h1ffB(cJ;^TIllfi-r5PlSBili6i5 

[0014] it^Jssw^u^^xVT'i^aasias^i 
(*, *xV7 , s£ictefBT5;><7V7'x-*©$»-f'y-\* 

y^j^ia^ 5i6n5^f^7f-$ ©is s 
[0015] m&m6(DT?)i>?*7-''(7 7 mmm8i>i5%; 

Itv *xV7'S£ai7D , HT&5;*xV7'l¥£3M8£u 7. 
h y-Ax-**J«#«!^S»Stl*7» h 'J -Ut—Z 

XhU-AlWBl^ftts buIB^ xV T'S^lft©;* xV 
T'HroPSB^PWtV h1ff?BlCcJ;-3TDifii5-r5lll^Ka6i5 

[0 0 16] W#^7©^U? : *xV7 7 llH!Sifi5£SH 



f8#Sf55ti3* xV 7f-^ t^-fA^'f >1f 58^6 
7. h U-i,7- S»*SfiKr*7. h U-iAX-^£fig#|g 
7.hy-^7 r -5«**«r*tc|»LTHI}m(IIII ! £15 
fc-5fc»©fc> hfciJIBT. h y-Ax-^fKf^ftK^- 

[0 0 17] n$B8©7/l/?p<xV7'imiliM 

J*. if^7IBt8W^;l/ i ? 1 p«x-f7'^i)ifiiJ^5£(iM 
{Cv ^xVFWSajafcfffcSyTVTTigE^a^ 7. 

fciiufB* x-f 7*»£ #«?SifeT»* * lCfttR?-« 
7 h U-/*ft?B!#S:<!:. 1513/ f-f T'lS^IS©;* x-f 
7*IB©HJH*HJHk> h1f?BlcJ;oTiifp-r5li)WPS5 
*S«-*U 7.hy-Z»x-*toa#g#»SfSStl3X 
h'J-^7- >*JWi!U 7. MJ -At 1 -* «*J«T* 

VlS8HcftoT>7 ? -f'7*lf£*fffcl.\ |5]JIBt > Hf IB 

t $ u=7 << >ma*i!Rd LTitenfflimp^s^^ x 

•< 7'i?£#iaK33tt5 ^7< 7'S*©|5l^Pli5^t : T^: 3 
[0 0 18] toft^9<»^)\,^*T<7mmsm&m. 
VhlSSfi^, ^7-f7f-^fifMtIt5^-fi, 

[0 0 19] i£d?Jl1 0©?;b9 1 yx-<7'^ISIi5^H 
t*. if5R3S9IBt8©^/Ux^7 :: ^7'PSHDigi5^aiiM 

ic, *v : <f7 , n$.iitt*firz5*T'C7n^mt, 7. 

^SuIB^xi'7 7 ^#STB^T'*-57 : '-^lc(e§!-r?> 
7.h'J-A»BS#Sd:x MIB^x-i'T'S^ISro^x-r 
7'IB©BMl*RIIJit> Hf«fcJ:-3TBIB-r*WIWIi«S 
^S^WL, tulBXK'J-iU^gB^eit^^-fL^-f 

»4©mm!IMIS«15fl:3#««li?lfeC t«W«t LT 

[0020] it^jsi 1 ©^/i/xyx-f Tmmmm^m. 
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7*»J£#h** 6ibn^/TV7f-^ ©raffle s 

[002 1] (f 1 2 (0^/1/** T'raffiDSffSB 
[0 0 2 2] 

[0 0 2 3] lfARJI2 0^l/5 1 ^x^7 7 !W]^illlS[5^5iv 
aO : if^8IBigco^;U^^5 1 V7 7 |Wl^i)IS5^S^ ± 

Irc^&tfS;*^ T'SOx-^ LT * =r-i 7W± 
«tcfii**Blwai«s*tf3a:3o 

[0024] m&m3<D^)[,T*T*7mmmwj5m. 

ffi t > h ««»ffi#«3B« ;* T 7 x- * fiStftlt 

TnfflHMis*f7a a tf fii^a c t ic <fe 

[0025] n&m4a>v)\,**T47mimttiims 

±E<0«rtW«:*y. W*3H3(3!)3&», Sfcttn#JB9CD 



^gtc£oT£/#i*ft/cXhy-Ax-££ffll N Ts £ 
H»BHB#IB^<fc y * xV ZB^KtcfeW-SRIfflWIlS 
[0 0 2 6] ff5RiI5CD^/l/5 1 >7 ; V7 7 |f3^1/ISi5^;£, 

=r- * gs^sfr %&mft*T*7T-z<D : r-* ^se 
*-*i*BM8t> hif$B>*scoit mt Lzmmty hum 

[0 0 2 7] ii5RJl6CD^;b^yx^7 7 |Wl^lflS5^;S, 
SUWSUSI 2E»0)-7;U^p«7 r ^7 7 HI»BHffiKHttx 
±EO«fiKti:J:y, If sRH 5 SfcttttSfcai 1 

<0l£Blc<fcoT£jffi*n/cX h 'J-Af-^^l^ 
ffiffl 5- n^> * 7 t - * <0 £ -f 7^ ftfcHJB! 

[0 0 2 8] 

[0029] (ni»9j D mi tmxm i ebjcd^i/^ 
[0030] mM^i^ziKD^i^^T-^Tmmmm 

* * t 7 x - * «k * -< a 5 *r > it n ^ e x h y - ^ x 

-^^4^5Xh'J-/.7-^MfSl Its P?£ 
*^«T«lcPRLTHIHIWI5«ff3B:3fe»fl!)t> h^X 

[00 3 1 ] E12t*if*Jl2IBi6C0^U^y7 r <7 7 ^ 

[0 0 3 2] E)2tCfc^TC:C0V;U^^xV7 7 fWJ^g5 

1 1 T^Z^tl/cX h 'J-A7-^^^tV7M^ 

2 2 ^- ^ \zmmt z> x h y -lmwzsbl 
2 x h y-Zx^-^^^-r^Ktc^-xetifci^i 
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[0 0 3 3] H3t4*SI«Sff!la)^;l/^y7 :f .<7 7 BI«BiBKi 
[0 0 3 4] coftWix fcMfet-r^jl/^y-rT 7 * 

[0 0 3 5] S/Xf^iLm EUlCjjVTcfc'plC, C 

puiooi, /x-Kx-fx^t, ninfi9»£«io 

1^ *^7<>tiSl0 2ts 7/bf^fV7f 
-^ROM103i:, *-7/l/W7h7-*t©U 
ARM 0 4£, mJBfcVhflBBA*i»1 2£, fHEfllR 

ami 0 6t, xhy-juy-^awi it, xhj 

-A^-£ft?I^SP2 1 <t, DRAM2 2 5<h, VRAM 
2 2 6, 7^X7U^f**-7-f*r/W^^7-< 
7 7 ^*SP2 2t, IBMBBBS92 3tT«a*nT^*o 
[00 36] ft, HK EI2<O^U-/xia<rj1 1^62 

[0 0 3 7] &tc, ±IB§gP<D5"6^35:^<rj(rjrt§tco 

^T«asKwr*o c p u i o 014, *mffiMW)\s^ 
[0038] mmm*m±m 101B, hi, 0271* 

[00 39] ^-fZ^f V«3Sl 0 3140K 02CO 

[0 0 4 0] £1414, *^Z*^>*»£*-£fc»U:ffi 
V-f -7 x -X(D«jSiiffi<&«l*jS L 

U 4 1 0, 4 2 0, 4 3 Ott^ft^tlfcTr^ ^-x 

-7Vl/T&£o 4 11, 4 12, 4 1 SliSastf^* 
X-^S-Sctt-TStiOTfey. 42 1, 4 2 2, 4 2 
3, 4 3 1 BS45*-7-f tr-**! St5o 4 3 
2144 3 1 tmC*-T«#7 : -*Z*1o%>o Z.O^'tU 
7<>«oT7ib7^7V7Km^, SMt 
0TtW4 1 1 7 r <7J-4 2 1 tfX£- m P$ 

pat i^siinT*— ^-f *4 3 i j&M--7 r -f*-4 2 1 

Klfi^cfc^tcLTX^- h-T^o BSfflt 2T?tT7 :r *'S 
4^4 1 2K^bt>, (Bj^lCTj— tt4 2 2 
oT4 3 1tt#ikT3 0 tt^TWBt 3*Stl7?|-SS 

a^tt») , 7^^**44 2 3 Jc^toSo ^t47- 
try* 4 1 3<DH£#*&£tU *— ?<*4 3 2tf*- 



x^*4 2 3tC«3-oTX^»hL, «*lCB38B t 5 T* 
[004 1] V/Uy^xVT'x-^fflROM 1 0 314, 

[0 0 4 2] ^-^Vk *y by-ttnmm&l 0 4 
14, C ^ 7i bf ^ 7> 7 7 - ^ ^111^ 

[0 0 4 3] tt±^Uy*y>f T^—^JBROM 1 0 3 
tmm^^ 0 414, IK 0 2lC*>^T;*7M'7 7 y-£ 

wi#sn oictg^-r^o 

[0 0 4 4] 2-VO*7i-710 5B % 
fi»1f A*>T l * * <P- KC0 cfc -5 £ -f > * 7 x - 

[0 0 4 5] HI, H2 0EIWfc>HIt«A»*«te« 

ar*rawk>h««A»»i 2«, ca>i«BA*i*s 

[0 0 4 6] f^HlRAMI 0 614, ±i3CPU 1 0 0 

[0 0 4 7] HK H20)Xh'J-i*7 r -4rSd6#«ic: 
ffla-r^Xh'J-jUy-^^figgl 7 ! 114, IQffiflmgS 

Mi o 1 , z^-rvesan o 2, ^yu^^^VT 7 

7-^lROM1 0 3£rtgfflVl 0 4, |B»kV h1ff« 
A*Jffl1 2^6coy-^, ItSB^SX h 'J-Zx7-^^ 

[0 0 4 8] m2<D7.hv-LxT-*mm^mzi®^? 

A,/cX h U -Zx^-* yV 7 7 §lcy- ^ 

[0 0 4 9] H2<D*y<7 r W£*«U:fflSr*p<7V 

^97/uxt\ flWAary-f xru-f^xtf-*-* 

[0 0 5 0] ^2(^^i||g5#S : tcffla-r^|Wl^gpg| 7 
2 314, BulBlRMfi«*t«£« 1 0 1 Q«ItSS/77A 

^s^y>r7 7 #cr)^p^<7CD-rn^yx^<7L, 

[00 5 1] 05(4, Xhy-Z*x-*£*WD»tt* 
[0 0 5 2] Jj(Ts H4tC©H=fe#BBLT«lfWlffilS 

(ai) **rz^>r>«3an o 2 ^eiwi^p< y-rT 7 ^ 

[0 0 5 3] (a 2) X h'J-Xxy-^Sfiggpi 1 14£ 
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[0 0 5 4] (a 3) It*. 

KSf (0a-+f 7 x - X=£51 LTBJSATrr *Sf(D 
[0 0 5 5] (a 4) a307f777»6ftftH)IBk 

[0 0 5 6] (a 5) a 4CD;*^y yTffft^fcWWlC 

*y, ^hy-^y-^^^-^wwisassTSo 

[0 0 5 7] (a 6) a 5 C0X5^ 7T>*£ L/ct 1 -^ 

[0 0 5 8] i*fl*«^T±E©X h 'J-/*f-^t 

7, 0 8, 09, 010, Ell 1 , 012, 013, 0 
1 4tB^TRWT5, 

[0 0 5 9] HStefcl^T* (a) t*|W)81fc:> KWIB* 

cdt 1 - * tc*t ltssbsjc * 3 anaastf ffat>h« 
[0060] (b) i* % x hu-Ax— stc^sns 

raMBkVh*^Lfcfl!IT*ys (a) Tm^tcT-tCD 
[0 06 1] COHte*5tt£»8iJ?<!:M\ HJfi-ffttfS 

ttlc:7p^/*£ LT-5-*.£>ft**\ £fd*XHJ-A 
^-*£»S^r*MT:/P^/*<t LTfflta^o £ 

[0 0 6 2] IHttWftW^flWtttMgijyt^C^-* 

te^*Sh*BI»fc:y hT, Kgy? r e p e a 

y£U5£*n**n^-cDfi*#i o»T**sct 

[006 3] 07, 08, 09, 01 0tC, H6<D*0T 
[0 0 6 4] H 7 ttMEU?* r epea t col^Kff&to 



*(D* PtV L/c y , Jf^Lfcy LTfft^x 
[0065] El 8 l*M»J?tf m o n o (DBSlCff&foftS 

[0 0 6 6] H9(iKS!i?tf b I a c k (WWcfiftfoti 

[0 0 6 7] HI OttK&J^tfqu i e tOBICffftto 
^iiiSCDS^IKP^cD^^^^^cttc^oTteco^T 14 

[o o 6 8] mwrntzcomz^pLztiztf, z.<dw 
Ti*K y sa-j* 4 o/ettm ltbmb Lfc 0 

[0 0 6 9] mwm#£L:fc»&u\ W^cOt 5 -* 
IcRIMBtiVhcDIBgiJ^jb^^A^CIft^T, 07~01 

[0070] HI 1, 012, 01 3. 0141*. 07 
-0 1 0 TKW LfcPlKBWffi»f¥®5IIBa>y T 7 *^ 

BSSW Wtc til p< x < T 7 *S L fc £ <f h => 4 

xDrnTrnm^ LfdaiTa&So 

[0 07 1 ] tK t 2, t 3, t 4tt»5WSOI«$iJ 
*9ci*-r«o 0 1 1 t*§gfcg'J^tf repeat tcvtcftCD 
»S*</x^^f >=&^LfctiO>T, alt*, r-f 

T£cfc3K18Etfnfc£>r/x5<>T*y* a2t*, a 
1 ^H^Lfc^tC^-^VTj-^-^^X^- h^jltlT 
IHWrnS^UfcttffiSSL, a3t*, PlfflHtEttft:* 

[0 0 7 2] <%, 01 1 (Da 2<D£?[^ t|— rVtf 
-^Atfii 1 <7> a 1 <7>£-f ^>TVluHEtfft/cB39J 
t 1 icx^-h-yr-rtc, iinTWSJt 2lcte£oTHi|l$B 

[0073] c<t>b§, m6(Dmmt>\-mmc*z>t. 

^Attrau^p^-fcoiisyigLTasy, 

TSCttCcfeoTPWIBBytffTaASo 01 1C9a2£ 
^t^—r-f^^-^AOHtlieHtt (t2-t1) 

^T£ 0 OfUi707P-ft- htC^Tl*<S5>^<h, 
n = 2<t^:oTs nilSWft^-^-f^y-^ttjgtlT 
t^<DT% «kyiSLp«P7 :r <*2|Hl*y h-TSCt^ft 
cntCctoTs 01 1 CDa 3 0)«J:3^2*ffi(Dp<n 

[0 0 7 4] 01 2 liHiJSU^m o n ofc-3ftRf©SS 
JU^-f V^LfefcOTs bit*, -rV^-^- 
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ct ? tciS^ zm/c 2 /a ^ < >T £ U , b2(4, b1£ 
WSUfctti^L, b3(t IQMIinVttf^Cffft 

B&2-Z><D!i$->{ZftfrnT^2><DlZs WSJt 1~t 
3$W$ifS^^ BSSJt 3-t SttfiJSfrO) 

[0 0 7 6] ^\ ESI 1 20Qb 2<D£z>[Z s *— x-f*^ 

t HC**-Hmc, MiTPWJt 2(Cte£oT(WiaB 

[0077] z<d$s s mecomwtyt-mmz&zt. 

*- x-f * =r - * B (4 * P x -f - 6 & cfc 3 
HBsSffft^So 01 20)b 2 -t**? - 

zscomftvsmi* (t 2- t 1 ) -easy* hi 20b 3 

«f**7hU Ell 20b KOcfe^tcWS'Jt 3T*1 

[0 07 8] 01 3 Itmftfrtf black ©BSCDBS* 
^L^^V^L/cfcOT** c 1 (4, tT-xV^^-^ 

3lcR£Snfc*<Z*^>T*sy. c2(4, cl^H 
SLfc^Ktf-xV *x-£(DX£- htfim^f 

[0 0 7 9] lfx*x-*AU:fi^Tfll^fi^B 

[0 0 8 0] <%, 01 3(Oc 2©J;3fc ?T-fV^x' 

e#0i 3coc 1 o*</x^<vtci9Sffnrc^»j 

t Hcx^-h-tf-fU:, 3BftTfl3»Jt 2(cte$oTI^]J8 
[0 08 1] £CQI$ V 06CDjf]gSfc> h1f^(Ccfc£<!:, 
£0 IHffiWfcCOtr^x — * A*NtA,T*l/>*<7>T\ 

SSirffto^x-rT 7 (cco^jt^^-x^x-^e) 

p^ (t2-t 1) /ct*ffit-rssas*?ift3o -r**:, 
in 1 3 <d c 3 o izm&mm<Dm&mmi$ ( t 2 - 1 

1) «ttfi<0:o^ ««s kT^x— »0)B*ttc 1 

coiSf^ J: y ( t 2 - 1 1 ) fcitoBSpaanrffftfen* 



-* e ttBHH3!?ttnT, fosiit t 5 ct y ( t 2 - 

t 1) f£frtH*lfcBS9Jt 5' T**l7f£ 0 

[0 0 8 2] 01 4l*BE»J?q u i e t(0^OflS*< 
Z^^V*3*Lfc*>(DT\ dm, tV^x— *F 
ttr*f"*BtfliSat 1 TlWJ^tCX^- ht^J:5 
(Cl8££ftfc£<A5>f >T'*y, d 2(4, d1^H?T 
Lfcl£U:*-x-f ^x-^OX*- htf3BtiTRI«B-rn 
#*i;fctttt*«U d3(4, WWBIttaif**f5ft-3T 
d 2T*Stlfcrtl**aLfctt«*iH-. 

[0 0 8 3] tfxJfx— ^BtCfc^T^SJ t 3~ 

[0 0 8 4] 01 4CDd 2<D£?iZ s 
-£Ftf01 4<7)d 1 <D^^U^^y[zWL^ntz^m 
t 1 tcx^-h^-THCs «tiT«*Jt 2lcJ6*oTlflW 

[0085] cob, mz><n^m\i>\*mmz&%>£^ 
B[zimmmB^mrmi*cD'piz^mm& 

&1±?%o (B»i] t 3 - t 4) ^\ HHWfcflDtfT 5 *^ 
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(54) DEVICE FOR CONTROLLING SYNCHRONIZATION MULTIMEDIA 

(57)Abstract: 

PURPOSE: To attain synchronous reproducting of plural media by appling 
information for controlling a synchronous lag between plural media whose 
generation is supposed at the time of reproducing data previously at the time of 
preparing stream data obtained by collecting the data of plural media such as 
audio and video devices as one data file based upon time series control. 
CONSTITUTION: This multimedia synchronous controlling device is provided 



with a medium data managing means 10, a stream data generating means 11 for 
generating stream data from medium data obtained from the means 10 and time 
line information and a synchronous hint information inputting means 12. At the 
time of generating stream data from a necessary medium data by the means 1 1 
in accordance with the time line information, hint information for executing control 
so as to accurately synchronize the reproducing of media is applied from the 
means 12 to the means 1 1 . 
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CLAIMS 



[Claim(s)] 

[Claim 1] A media data control means to manage data and the time line of 
multimedia, The media data obtained from said media data control means, and a 
stream data generation means to generate stream data from time-line 
information, It has a synchronous hint information input means to give the hint for 
performing synchronizing hold control to the stream data generated. The time- 
line information which said media data control means owns is followed. In case 
said stream data generation means generates stream data from required media 
data The multimedia synchronizing-hold-control approach characterized by a 
synchronous hint information input means giving by making into synchronous hint 
information information on the contents of actuation performed in order to adjust 
the synchronous gap between the media which may be produced at the time of 
playback, the data set as the object of accommodation. 
[Claim 2] A media playback means to perform a media playback output in the 
multimedia synchronizing-hold-control approach according to claim 1, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media at the time of playback using synchronous hint information. The stream 
data obtained from a stream data generation means are decoded. The 
synchronous hint information given when generating stream data, and data with 
each refreshable media, Decompose into time-line information and said media 
playback means performs media playback according to this time-line information. 
The multimedia synchronizing-hold-control approach characterized by collating 
synchronous hint information and time-line information, and said synchronizing- 
hold-control means performing synchronizing hold control of the media playback 
in a media playback means. 

[Claim 3] The multimedia synchronizing-hold-control approach according to claim 
1 characterized by having a synchronous hint information decision means to 
determine the synchronous hint information for performing synchronizing hold 



control with reference to the time-line schedule which a media data control 
means manages in the case of media playback, making it connected with time- 
line information in the case of stream data generation, and giving a synchronous 
hint. 

[Claim 4] A media playback means to perform a media playback output in the 
multimedia synchronizing-hold-control approach according to claim 3, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], The 
multimedia synchronizing-hold-control approach characterized by performing 
synchronizing hold control of media playback according to the synchronous hint 
information determined from the time-line information which has a synchronizing- 
hold-control means to adjust the synchronization between the media at the time 
of playback using synchronous hint information, and is acquired from said stream 
decode means. 

[Claim 5] Said synchronous hint information decision means determine the 
synchronous hint information for having a data-type judging means to judge the 
type and capacity of the media data used for media playback, and performing 
synchronizing hold control in the case of media playback is the multimedia 
synchronizing-hold-control approach according to claim 3 characterized by what 
it opts for from the information on the media data obtained from said data-type 
judging means. 

[Claim 6] A media playback means to perform a media playback output in the 
multimedia synchronizing-hold-control approach according to claim 5, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], The 
multimedia synchronizing-hold-control approach characterized by performing 
synchronizing hold control of media playback according to the synchronous hint 
information determined by the data type which has a synchronizing-hold-control 
means to adjust the synchronization between the media at the time of playback 
using synchronous hint information, and is obtained from said stream decode 



means. 

[Claim 7] A media data control means to manage data and the time line of 
multimedia, The media data obtained from said media data control means, and a 
stream data generation means to generate stream data from time-line 
information, It has a synchronous hint information input means to give the hint for 
performing synchronizing hold control to the stream data generated. The time- 
line information which said media data control means owns is followed. In case 
said stream data generation means generates stream data from required media 
data Multimedia synchronizing-hold-control equipment characterized by having 
the means which a synchronous hint information input means gives by making 
into synchronous hint information information on the contents of actuation 
performed in order to adjust the synchronous gap between the media which may 
be produced at the time of playback, the data set as the object of 
accommodation. 

[Claim 8] A media playback means to perform a media playback output in 
multimedia synchronizing-hold-control equipment according to claim 7, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media at the time of playback using synchronous hint information. The stream 
data obtained from a stream data generation means are decoded. The 
synchronous hint information given when generating stream data, and data with 
each refreshable media, Decompose into time-line information and said media 
playback means performs media playback according to this time-line information. 
Multimedia synchronizing-hold-control equipment characterized by having a 
means by which collate synchronous hint information and time-line information, 
and said synchronizing-hold-control means performs synchronizing hold control 
of the media playback in a media playback means. 

[Claim 9] Multimedia synchronizing-hold-control equipment according to claim 1 
characterized by having a means to have a synchronous hint information 



decision means to determine the synchronous hint information for performing 
synchronizing hold control with reference to the time-line schedule which a media 
data control means manages in the case of media playback, to make it 
connected with time-line information in the case of stream data generation, and 
to give a synchronous hint. 

[Claim 10] It sets to multimedia synchronizing-hold-control equipment according 
to claim 9, a media playback means to perform a media playback output, A 
stream decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media at the time of playback using synchronous hint information. Multimedia 
synchronizing-hold-control equipment characterized by having a means to 
perform synchronizing hold control of media playback according to the 
synchronous hint information determined from the time-line information acquired 
from said stream decode means. 

[Claim 11] Said synchronous hint information decision means determine the 
synchronous hint information for having a data-type judging means judge the 
type and the capacity of the media data used for media playback, and performing 
synchronizing hold control in the case of media playback is multimedia 
synchronizing-hold-control equipment according to claim 3 characterized by to 
have a means determine from the information on the media data obtained from 
said data-type judging means. 

[Claim 12] A media playback means to perform a media playback output in 
multimedia synchronizing-hold-control equipment according to claim 11, A 
stream decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media at the time of playback using synchronous hint information. Multimedia 
synchronizing-hold-control equipment characterized by having a means to 
perform synchronizing hold control of media playback according to the 



synchronous hint information determined by the data type obtained from said 
stream decode means. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention controls two or more media serially, and 
relates to the multimedia equipment reproduced while taking the synchronization 
between media. 
[0002] 

[Description of the Prior Art] In the field of information processing, a multimedia 
regenerative apparatus which reproduces two or more media called an audio and 
video to coincidence includes the radical of time control broadly, and came to be 
used for it in recent years. 

[0003] The personal multimedia regenerative apparatus using the data storage 
means by optical disk like CD-ROM as an example which generally came to be 
used recently can be considered. Although media data, such as voice, video, a 
still picture, and a document, are taken out from CD-ROM and they are 
reproduced on a display or a loudspeaker with such equipment, these media are 
doubled with time amount advance, and it sometimes often reproduces to 
coincidence. 

[0004] There are an approach of being a call in specifying the address of the data 
arranged in on CD-ROM variously [ the approach of time control ] suitably but, for 
example by the program, and performing time control, and an approach CD-ROM 
interleaved according to the time order foreword defined beforehand performs 
serial synchronous playback, and synchronous playback is performed on the 
radical of a time schedule. 



[0005] In order to realize such synchronous playback, there is data format of the 
stream data which two or more media are interleaved [ data ] to record media, 
such as CD-ROM, as one data, and make them record on them. 
[0006] 

[Problem(s) to be Solved by the Invention] However, with the above-mentioned 
conventional technique, when access of data and acquisition were performed in 
response to an instruction of media playback, two or more media were 
reproduced by coincidence and the load which exceeded permissible capacity 
depending on the type and capacity of data was applied to CPU of a system, a 
gap arises in the set - up playback time amount, and a synchronization might be 
unable to be taken between media. 

[0007] Such a condition is briefly explained using drawing 25 . Drawing 25 is the 
time-line Fig. which expressed the media reproduced according to advance of 
time amount, and data in ** type, and presupposes that media playback was set 
up like A of this drawing. An audio and video are set up here so that any data 
may be started to coincidence in synchronous Rhine. However, if this is 
reproduced, it is such remarkable delay that slight delay becomes behind at the 
beginning which delay may produce to one media and was produced in the audio 
in the condition like B of this drawing. 

[0008] In order to solve such a situation, the approach of devising the data list in 
record media, such as CD-ROM, is taken, but since the stream data with which 
two or more data are recorded as one data are also controlled independently for 
every media in the case of media playback, it is not the dissolution of a perfect 
synchronous gap in many cases. Moreover, in order to require time amount for 
performing synchronizing hold control in consideration of a data type at the time 
of playback, smooth synchronizing hold control was difficult 
[0009] It aims at offering the approach of always securing the synchronicity 
between media, in view of the above-mentioned problem in this invention. 
[0010] 

[Means for Solving the Problem] In order to attain the above-mentioned purpose, 



the multimedia synchronizing-hold-control approach of claim 1 A media data 
control means to manage data and the time line of multimedia, The media data 
obtained from said media data control means, and a stream data generation 
means to generate stream data from time-line information, It has a synchronous 
hint information input means to give the hint for facing generating stream data 
and performing synchronizing hold control to said stream data generation means. 
According to the time line which said media data control section owns, said 
stream data generation means generates stream data from required media data. 
It is characterized by giving the synchronous hint information for performing 
accommodation which can take the synchronization between media correctly at 
the time of playback in that case. 

[0011] A media playback means by which the multimedia synchronizing-hold- 
control approach according to claim 1 performs a media playback output further 
in the multimedia synchronizing-hold-control approach of claim 2, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media of said media playback means using synchronous hint information. The 
stream data obtained from a stream data generation means are decoded. With 
the synchronous hint information given when generating stream data, and a 
media playback means, data with each refreshable media, It is characterized by 
decomposing into time-line information, and for said media playback means 
performing media playback according to this time-line information, collating 
synchronous hint information and time-line information, and said synchronizing- 
hold-control means performing synchronizing hold control of the media playback 
in a media playback means. 

[0012] Multimedia synchronous playback accommodation of claim 3 has a 
synchronous hint information decision means to determine the synchronous hint 
information for performing synchronizing hold control with reference to the time- 
line schedule which a media data control means manages in the case of media 



playback, and is characterized by giving a synchronous hint with time-line 
information in the case of stream data generation. 

[0013] A media playback means by which the multimedia synchronizing-hold- 
control approach according to claim 3 performs a media playback output further 
in the multimedia synchronizing-hold-control approach of claim 4, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media of said media playback means using synchronous hint information, and is 
characterized by performing synchronizing hold control of media playback 
according to the synchronous hint information determined from the time-line 
information acquired from said stream decode section. 
[0014] The multimedia synchronizing-hold-control approach of claim 5 has a 
data-type judging means judge the type and capacity of the media data used for 
media playback, and said synchronous hint information decision means 
determine the synchronous hint information for performing synchronizing hold 
control in the case of media playback is characterized by what it opts for from the 
information on the media data obtained from said data-type judging means. 
[0015] A media playback means by which the multimedia synchronous playback 
approach according to claim 5 performs a media playback output further in the 
multimedia synchronizing-hold-control approach of claim 6, A stream decode 
means to decode the stream data obtained from a stream data generation means 
to data reproducible [ with said media playback means ], It has a synchronizing- 
hold-control means to adjust the synchronization between the media of said 
media playback means using synchronous hint information, and is characterized 
by performing synchronizing hold control of media playback according to the 
synchronous hint information determined by the data type obtained from said 
stream decode section. 

[0016] A media data control means by which the multimedia synchronizing-hold- 
control equipment of claim 7 manages data and the time line of multimedia, The 



media data obtained from said media data control means, and a stream data 
generation means to generate stream data from time-line information, It has a 
synchronous hint information input means to give the hint for facing generating 
stream data and performing synchronizing hold control to said stream data 
generation means. According to the time line which said media data control 
section owns, said stream data generation means generates stream data from 
required media data. It is characterized by having a means to give the 
synchronous hint information for performing accommodation which can take the 
synchronization between media correctly at the time of playback in that case. 
[0017] A media playback means by which the multimedia synchronizing-hold- 
control approach according to claim 7 performs a media playback output further 
in the multimedia synchronizing-hold-control equipment of claim 8, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media of said media playback means using synchronous hint information. The 
stream data obtained from a stream data generation means are decoded. With 
the synchronous hint information given when generating stream data, and a 
media playback means, data with each refreshable media, Decompose into time- 
line information and said media playback means performs media playback 
according to this time-line information. It is characterized by having a means by 
which collate synchronous hint information and time-line information, and said 
synchronizing-hold-control means performs synchronizing hold control of the 
media playback in a media playback means. 

[0018] The multimedia synchronizing-hold-control equipment of claim 9 has a 
synchronous hint information decision means to determine the synchronous hint 
information for performing synchronizing hold control with reference to the time- 
line schedule which a media data control means manages in the case of media 
playback, and is characterized by having a means to give a synchronous hint 
with time-line information in the case of stream data generation. 



[0019] The multimedia synchronizing-hold-control equipment of claim 10 A media 
playback means by which the multimedia synchronizing-hold-control approach 
according to claim 9 performs a media playback output further, A stream decode 
means to decode the stream data obtained from a stream data generation means 
to data reproducible [ with said media playback means ], It has a synchronizing- 
hold-control means to adjust the synchronization between the media of said 
media playback means using synchronous hint information, and is characterized 
by having a means to perform synchronizing hold control of media playback 
according to the synchronous hint information determined from the time-line 
information acquired from said stream decode section. 

[0020] The multimedia synchronizing-hold-control equipment of claim 1 1 has a 
data-type judging means judge the type and the capacity of the media data used 
for media playback, and said synchronous hint information decision means 
determine the synchronous hint information for performing synchronizing hold 
control in the case of media playback is characterized by to have a means 
determine from the information on the media data obtained from said data-type 
judging means. 

[0021] The multimedia synchronizing-hold-control equipment of claim 12 A media 
playback means by which the multimedia synchronizing-hold-control approach 
according to claim 1 1 performs a media playback output further, A stream 
decode means to decode the stream data obtained from a stream data 
generation means to data reproducible [ with said media playback means ], It has 
a synchronizing-hold-control means to adjust the synchronization between the 
media of said media playback means using synchronous hint information, and is 
characterized by having a means to perform synchronizing hold control of media 
playback according to the synchronous hint information determined by the data 
type obtained from said stream decode section. 
[0022] 

[Function] The multimedia synchronizing-hold-control approach and the 
multimedia synchronizing-hold-control equipment according to claim 7 of claim 1 



generate stream data by the above-mentioned configuration from the time-line 
information to which the schedule of the media data with which the media data 
control section has managed the stream data generation means, and serial 
media playback was set. In order to cancel the synchronous gap considered to 
produce a synchronous hint information input means at this time when this 
stream data is actually reproduced, the stream data which realize little 
synchronous playback of a time gap are generated by receiving the hint for 
accommodation, such as to which timing to perform synchronizing hold control, 
from the exterior, and giving a stream data generation means. 
[0023] The multimedia synchronizing-hold-control approach and the multimedia 
synchronizing-hold-control equipment according to claim 8 of claim 2 send the 
time-line information and synchronous hint information which the stream decode 
means disassembled into the data for every media the stream data generated by 
the approach of claim 1 , or the equipment of claim 7, and were read in delivery 
and stream data for the media playback means by the above-mentioned 
configuration to a synchronizing-hold-control means. The synchronizing-hold- 
control means which received synchronous hint information performs 
synchronizing hold control in a media playback means based on synchronous 
hint information, referring to the clock of the whole system which a media data 
control means manages. 

[0024] The multimedia synchronizing-hold-control approach and the multimedia 
synchronizing-hold-control equipment according to claim 9 of claim 3 By the 
above-mentioned configuration, a synchronous hint information decision means 
refers to the time line for the media playback which the media data control 
section manages, without giving the hint for carrying out synchronizing hold 
control at the time of reproducing stream data by the external input. By 
presuming where synchronizing hold control should be performed, synchronous 
hint information is determined and the stream data which realize little 
synchronous playback of a time gap are generated. 

[0025] The multimedia synchronizing-hold-control approach and the multimedia 



synchronizing-hold-control equipment according to claim 10 of claim 4 perform 
synchronizing hold control in a media playback means with the synchronizing- 
hold-control means which read the synchronous hint determined from time-line 
information by the above-mentioned configuration using the stream data 
generated by the approach of claim 3, or the equipment of claim 9. 
[0026] The multimedia synchronizing-hold-control approach and the multimedia 
synchronizing-hold-control equipment according to claim 1 1 of claim 5 By the 
above-mentioned configuration, a data-type judging means investigates the data 
format and capacity of required media data from a media data control means, 
and takes those information into consideration. The load time of day of data 
which a synchronous gap does not produce at the time of playback is deduced, 
and the stream data which realize synchronous playback from which 
synchronous hint information is determined as information on synchronous hint 
information decision, and a time gap does not produce it are generated. 
[0027] The multimedia synchronizing-hold-control approach and the multimedia 
synchronizing-hold-control equipment according to claim 12 of claim 6 perform 
synchronizing hold control [ in / in the synchronizing-hold-control means which 
read the synchronous hint determined from the type of the media data used by 
the above-mentioned configuration using the stream data generated by the 
approach of claim 5, or the equipment of claim 1 1 / a media playback means ]. 
[0028] 

[Example] Hereafter, the example of the multimedia synchronizing-hold-control 
equipment concerning this invention is explained to a detail, referring to a 
drawing. 

[0029] (Example 1) Drawing 1 is the multimedia synchronizing-hold-control 
approach according to claim 1 and the claim Fig. showing the configuration of 
multimedia synchronizing-hold-control equipment according to claim 7. 
[0030] In drawing 1 this multimedia synchronizing-hold-control approach and 
equipment A media data control means 10 to manage multimedia data and time 
line, such as an audio and video, The media data obtained from the media data 



control means 10, and a stream data generation means 1 1 to generate stream 
data from time-line information, It consists of synchronous hint information input 
means 12 to give the hint for facing generating the stream data which perform 
correctly the synchronization between the media at the time of playback, and 
performing synchronizing hold control to the stream data generation means 11. 
[0031] Drawing 2 is the multimedia synchronizing-hold-control approach 
according to claim 2 and the claim Fig. showing the configuration of multimedia 
synchronizing-hold-control equipment according to claim 8. 
[0032] In drawing 2 this multimedia synchronizing-hold-control approach and 
equipment In the configuration of the multimedia synchronizing-hold-control 
approach according to claim 1, or the configuration of multimedia synchronizing- 
hold-control equipment according to claim 7, in addition, a media playback 
means 22 to perform a media playback output, A stream decode means 21 to 
decode the stream data generated with the stream data generation means 1 1 to 
data reproducible [ with the media playback means 22 ], The synchronous hint 
information given when generating stream data is acquired from the stream 
decode means 21, and it consists of synchronizing-hold-control means 23 to 
adjust the synchronization between the media of the media playback means 22, 
referring to the time-line information which the media data control means 10 
manages. 

[0033] Drawing 3 is the block diagram of the multimedia synchronizing-hold- 
control approach of this example, and the whole system containing equipment. 
[0034] Although the target multimedia was limited to an audio and video and 
simplification of a configuration is planned in this example, it is also possible to 
be aimed at media other than this in practice. 

[0035] As shown in drawing 2 , as a system CPU 100 and a hard disk, The 
synchronizing signal generating machine 101, the time-line Management 
Department 102, and ROM103 for multimedia data, The connection 104 with a 
cable or a network, and the synchronous hint information input section 12, It 
consists of a working-level month RAM 106, the stream data generation section 



11, the stream data decode section 21, the media playback sections 22, such as 
DRAM225, VRAM226, a display, and an audio device, and the synchronizing- 
hold-control section 23. 

[0036] In addition, the same number is attached about the section corresponding 
to each means from 1 1 to 23 of the claim Fig. of drawing 1 and drawing 2 . 
[0037] Next, an approximate account is carried out about the contents of the 
main things among each part of the above. CPU100 performs control of the 
whole multimedia synchronizing-hold-control equipment of this example, data 
processing which needs each part. 

[0038] The synchronizing signal generating machine 101 is a system clock which 
accompanies the media data control means 10 and manages system-wide time 
amount and a synchronousr control in drawing 1 and drawing 2 . 
[0039] The time-line Management Department 103 exists in drawing 1 and the 
media data control means 10 of drawing 2 , and manages the time line at the 
time of media playback. The time line is the serial managed table in which the 
time schedule for performing synchronous playback of two or more media was 
described. 

[0040] It is what showed the example of the display screen of the user interface 
used in order that drawing 4 may set up the time line, and in this drawing, an axis 
of abscissa means advance of time amount, and 410, 420, and 430 are the 
tables which can set up the contents by which video, Audio A, and Audio B are 
reproduced, respectively. 411, 412, and 413 mean a different video data, and 
421, 422, 423, and 431 mean different audio data. 432 is the same audio data as 
431. If multimedia playback is performed according to this time line, video 41 1 
and an audio 421 will start by time amount to, it is late for time amount t1, and as 
an audio 431 covers an audio 421 , it will start. Video recovery changes to 412 by 
time amount t2, an audio also changes to coincidence 422, and 431 stops. Then, 
video recovery disappears from time amount t3 (for example, condition anything 
does not have a picture in one color in a playback screen), and an audio changes 
to 423. Playback of video 413 starts in time amount t4, an audio 432 covers an 



audio 423, and starts, and, finally all media are completed by time amount t5. 
[0041] ROM103 for multimedia data is a part which is storing multimedia data 
which were explanation of the time line, such as video and an audio, for example, 
can consider CDROM, LD, etc. 

[0042] The connection 104 with a cable and a network is used when obtaining 
the multimedia data used with this equipment from a communication network, a 
cable, etc. 

[0043] ROM103 for multimedia data and a connection 104 are above equivalent 
to the media data control means 10 in drawing 1 and drawing 2 . 
[0044] A user interface 105 is an interface device like the keyboard into which a 
user can input direct information. 

[0045] The synchronous hint information input section 12 equivalent to drawing 1 
and the synchronous hint information input means of drawing 2 outputs the 
synchronous hint information in the case of stream data generation in response 
to this information input. 

[0046] A working-level month RAM 106 is a working-level month field for the 
above CPU 100 to work. 

[0047] The stream data generation section 1 1 equivalent to drawing 1 and the 
stream data generation means of drawing 2 generates stream data from the data 
from the synchronizing signal generating machine 101, the time-line 
Management Department 102, ROM 103 for multimedia data or a connection 104, 
and the synchronous hint information input section 12, and information. 
[0048] The stream data decode section 21 equivalent to the stream data decode 
means of drawing 2 decodes stream data including synchronous hint information, 
and disassembles data for every media. 

[0049] The media playback section 22 equivalent to the media playback means 
of drawing 2 is the device which performs media playback according to the time 
line, for example, are a display, a loudspeaker, etc. 
[0050] If the synchronizing-hold-control section 23 equivalent to the 
synchronizing-hold-control means of drawing 2 checks a gap of the clock for 



every media in the media playback section 22 and has a gap, referring to the 
time amount of the whole system which said synchronizing signal generating 
machine 101 manages, it will update a clock based on synchronous hint 
information, and will perform synchronizing hold control. 
[0051] Drawing 5 shows the flow chart explaining actuation of the stream data 
generation section. 

[0052] Hereafter, with reference to this drawing, explanation of operation is given 
to drawing 4 . 

(a1) The stream data generation section 1 1 acquires the time-line information to 
which time series information required for synchronous media playback is set 
from the time-line Management Department 102. 

[0053] (a2) The stream data generation section 1 1 acquires the media data set 
up there with reference to the time line from the connection 104 with ROM 103 for 
multimedia data, a cable, and a network. 

[0054] (a3) The stream data generation section 1 1 acquires the synchronous hint 
information for maintaining the synchronization without a gap through the 
synchronous hint information input section 12, in case the data to create are 
reproduced. The input of hint information is performed by a user doing a direct 
input through user interfaces, such as a keyboard, etc. 
[0055] (a4) The synchronous hint information acquired at the step of a3 is 
checked with time-line information, and decision to which timing to perform 
synchronizing hold control, such as renewal of the clock of media, is performed. 
[0056] (a5) The decision performed at the step of a4 determines the data 
configuration of stream data. 

[0057] (a6) The data configuration for which it opted at the step of a5 generates 
stream data. 

[0058] The example of operation which the above-mentioned stream data 
generation section performs using an example and which gives a synchronous 
hint is explained using drawing 6 , drawing 7 , drawing 8 , drawing 9 , drawing 10 , 
drawing 1 1 , drawing 12 , drawing 13 , and drawing 14 . 



[0059] In drawing 6 , (a) is a showing [ the example of the data which give 
synchronous hint information ]-intelligibly thing, and it is shown what kind of 
accommodation is performed to each data at the time of playback. 
[0060] (b) is the example which showed the synchronous hint given to stream 
data, and has shown the synchronous hint information set up to the example of 
the data used by (a). It is given in forms, such as the structure, in fact. 
[0061] When a synchronous gap produces the identifier in this drawing, the 
stream data set as the object of that accommodation are given, and actuation of 
accommodation changes with identifiers. The pattern of actuation according to an 
identifier is prepared as a program by the side which is given as a program with 
stream data, or decodes stream data. Although the alphabetic word is used as an 
identifier in this example, usually it is provided by an ID number etc. in practice. 
[0062] When it is the audio data A with which the information on the section for 
accommodation is the synchronous hint given to data with an identifier, for 
example, Identifier repeat was given, it is given as synchronous hint information 
that the die length of the melody reproduced repeatedly continuously is 10 
seconds. Moreover, it is shown by the video data A that the time of day when the 
monochrome screen in which no pictures are reflected appears is for 30 seconds 
in 10 seconds and after 3 minutes and 10 seconds after [ of initiation of data ] 1 
minute. 

[0063] It was shown in what kind of process actuation of different synchronizing 
hold control for every identifier in which it used for drawing 7 , drawing 8 , 
drawing 9 , and drawing 10 in the example of drawing 6 is performed. 
[0064] Drawing 7 is the accommodation actuation performed when an identifier is 
repeat, reduces or increases the melody of the fixed die length reproduced 
repeatedly, and performs synchronizing hold control with other media here. 
[0065] Drawing 8 is the accommodation actuation performed when an identifier is 
mono, and performs synchronizing hold control with other media by changing the 
die length of the section when the monotone which does not have change in a 
tone is reproduced in this case. 



[0066] Drawing 9 is the accommodation actuation performed when an identifier is 
black, and performs synchronizing hold control with other media in this case by 
changing the die length of the section when the monochrome screen where no 
pictures are reproduced is displayed. 

[0067] Drawing 10 is the accommodation actuation performed when an identifier 
is quiet, and in this case, when a motion of a picture changes the die length of 
the display section of few screens at the time of playback, it performs 
synchronizing hold control with other media. 

[0068] Although the identifier was considered in addition to this, for the time 
being, only four, it was shown and this example explained it. 
[0069] When a synchronous gap arises, it confirms whether the identifier of a 
synchronous hint is in the data under playback, and accommodation actuation as 
shown by drawing 7 - drawing 10 is performed. If there is nothing, the clock of 
each media will be unified and adjusted compulsorily. 

[0070] Drawing 11 , drawing 12 , drawing 13 , and drawing 14 are the examples 
briefly shown in the form of the time line where what kind of accommodation is 
performed showed the media currently reproduced serially, when the 
synchronizing-hold-control actuation explained by drawing 7 - drawing 10 is 
applied at the time of actual media playback. 

[0071] t1 , t2, t3, and t4 mean a certain specific time of day. Drawing 1 1 is what 
showed the playback time line when an identifier is repeat, a1 is the time line set 
up so that the audio data A and a video data P might start to coincidence at time 
of day t1 , when a2 performs a1 , it expresses the condition that the start of audio 
data was overdue and the synchronous gap arose, and a3 expresses the 
condition canceled the gap which performed synchronizing-hold-control actuation 
and was seen by a2. 

[0072] A situation which is overdue, starts at time of day t2, without the audio 
data A starting like a2 of drawing 1 1 now at the time of day t1 set as the time line 
of a1 of drawing 1 1 , and a synchronous gap produces is considered. 
[0073] According to the synchronous hint information on drawing 6 at this time, 



the audio data A are the repeat of the same melody, and can perform 
synchronizing hold control by increasing or omitting some units of that melody. 
When a2 of drawing 1 1 is seen, the time delay of the audio data A is (t2-t1), and 
this is equivalent to two units of the melody repeated. That is, if it applies to the 
flow chart of drawing 7 , since it was set to n= 2 and the audio data for 
accommodation are behind, a repeat melody will be cut twice. The melody of two 
units will be cut like a3 of drawing 11 by this, and a synchronization will be taken 
by it. 

[0074] Drawing 12 is what showed the playback time line when an identifier is 
mono, b1 is the time line set up so that the audio data B and a video data Q 
might start to coincidence at time of day t1 , when b2 performs b1 , it expresses 
the condition that the start of audio data was overdue and the synchronous gap 
arose, and b3 expresses the condition canceled the gap which performed 
synchronizing-hold-control actuation and was seen by b2. 
[0075] In addition, even the time of day t1-t3 of being divided into the pattern 
whose rectangle which means the audio data B is two is monotone playback, and 
it means that time of day t3-t5 is data with which a certain music playback exists. 
[0076] A situation which is overdue, starts at time of day t2, without the audio 
data B starting like b2 of drawing 12 now at the time of day t1 set as the time line 
of b1 of drawing 12 , and a synchronous gap produces is considered. 
[0077] According to the synchronous hint information on drawing 6 at this time, 
the audio data B have the continuation section of a monotone which does not 
become a melody, and when only the time amount shifted in that section 
increases or cuts a monotone, synchronizing hold control can be performed. The 
time delay of the audio data B is (t2-t1), and if b2 of drawing 12 is seen, in b3 of 
drawing 12 , only the time amount of this die length will cut the monotone of the 
audio data B, and it will adjust it so that a monotone may be completed at time of 
day t3 like b1 of drawing 12 . If current time is tc, synchronizing hold control will 
be performed here and the clock of an audio will be set by the clock of video. 
[0078] Drawing 13 is what showed the playback time line in case an identifier is 



black, c1 is the time line set up so that the audio data E and a video data A might 
start to coincidence at time of day t1, when c2 performs c1 , it expresses the 
condition that the start of audio data was overdue and the synchronous gap 
arose, and c3 expresses the condition of having canceled the gap which 
performed synchronizing-hold-control actuation and was seen by c2. 
[0079] In addition, in a video data A, the black rectangle means the time amount 
section when the monochrome screen in which no pictures are reflected is 
reproduced. 

[0080] A situation which is overdue, starts at time of day t2, without the audio 
data E starting like c2 of drawing 13 now at the time of day t1 set as the time line 
of c1 of drawing 13 , and a synchronous gap produces is considered. 
[0081] According to the synchronous hint information on drawing 6 at this time, 
the monochrome screen in which synchronizing hold control is possible exists in 
a video data A. Since playback will be suspended [ since the video data A for 
accommodation is progressing now, ] until it can take other media (this example 
audio data E) and a synchronization if the flow chart of drawing 7 - drawing 10 is 
followed, and a monochrome screen starts, and a monochrome screen starts at 
time of day t3 in this case, processing only whose time amount (t2-t1) suspends 
playback of video here is performed. Then, the playback time amount of a 
monochrome screen becomes long [ (t2-t 1) ] like c3 of drawing 13 , and 
playback of a video data will be henceforth performed later than a setup of c1 the 
time of (t2-t1). Consequently, a video data A and the audio data E can take a 
synchronization, and end the playback by time-of-day t5' in which only (t2-t 1) 
was behind [ t5 ]. 

[0082] Drawing 14 is what showed the playback time line at the time of Identifier 
quiet, d1 is the time line set up so that the audio data F and a video data B might 
start to coincidence at time of day t1 , when d2 performs d1 , it expresses the 
condition that the start of audio data was overdue and the synchronous gap 
arose, and d3 expresses the condition of having canceled the gap which 
performed synchronizing-hold-control actuation and was seen by d2. 



[0083] In addition, in the video data B, it means that image reconstruction with 
few motions is performed in the time amount section in time of day t3-t4. 
[0084] A situation which is overdue, starts at time of day t2, without the audio 
data F starting like d2 of drawing 14 now at the time of day t1 set as the time line 
of d1 of drawing 14 , and a synchronous gap produces is considered. 
[0085] According to the synchronous hint information on drawing 6 at this time, 
an image with few motions in which synchronizing hold control is possible exists 
in a video data B. (Time of day t3-t4) Since playback will be suspended [ since 
the video data B for accommodation is progressing now, ] until it can take other 
media (this example audio data F) and a synchronization if the flow chart of 
drawing 7 - drawing 10 is followed, and a monochrome screen starts, and few 
accommodation screens of a motion at time of day t3 start in this case, 
processing between [ whose ] time amount (t2-t1) suspends playback of video 
here is performed. Then, the playback time amount of a screen with few motions 
becomes long [ (t2-t 1) ] like d3 of drawing 14 , and playback of a video data will 
be henceforth performed later than a setup of d1 the time of (t2-t1). 
Consequently, a video data B and the audio data F can take a synchronization, 
and end the playback by time-of-day t5' in which only (t2-t 1) was behind [ t5 ]. 
[0086] Drawing 15 shows the internal configuration of the stream data decode 
section of the multimedia synchronous regenerative apparatus of drawing 3 , the 
media playback section, and the synchronizing-hold-control section in more detail. 
[0087] As shown in drawing 15 , as a system the stream data decode section 21 
The stream data decomposition section 211 and the synchronous hint 
information output section 212, It consists of the time series information output 
section 213, the audio data output section 214, and the video-data output section 
215. The media playback section 22 The audio control section 221, the video 
control section 222, and the audio clock 223, It consists of the video recovery 
clock 224, DRAM225 and VRAM226, an audio device 227, and a display 228. 
The synchronizing-hold-control section 23 It consists of the time series 
information control section 231, the synchronous hint information recognition 



section 232, the time series information collating section 233, the synchronicity 
Monitoring Department 234 between media, the updating part cognitive section 
235, and the renewal section 236 of synchronicity. 

[0088] Moreover, although the target media are limited to an audio and video in 
this example of a system, it is also possible to prepare other data output sections 
and control sections about media. 

[0089] Next, an approximate account is carried out about the contents of the 
main things among each part of the above. The stream data decomposition 
section 21 1 decodes the inputted stream data, decomposes, and divides data for 
every media. 

[0090] The synchronous hint information output section 212 receives information 
required in order to perform synchronizing hold control at the time of media 
playback from said stream data decomposition section, and outputs it. 
[0091] The time series information output section 213 takes out the time series 
information set as the time line from the disassembled stream data, and passes it 
to the synchronizing-hold-control section. 

[0092] The audio data output section 214 picks out the data about an audio from 
the disassembled stream data, and passes them to the media playback section. 
[0093] The video-data playback section 215 picks out the data about video from 
the disassembled stream data, and passes them to the media playback section. 
[0094] The audio control section 221 receives the audio data obtained from the 
stream data decode section 21 , and controls an audio function to be able to 
perform audio playback based on time series information. 
[0095] The video control section 222 receives the video data obtained from the 
stream data decode section 21, and controls a video function to be able to 
perform video recovery based on a hour entry. 

[0096] The audio clock 223 is interlocked with playback of an audio, and 
performs time management of an audio. 

[0097] The video recovery clock 224 is interlocked with playback of video, and 
performs time management of video. 



[0098] The audio device 227 performs an audio playback output. A display 228 
performs a video recovery output. 

[0099] The time series information control section 231 controls the media 
playback in the media playback section according to the time series information 
on the time line obtained from said time series information output section with 
reference to the synchronizing signal which the synchronizing signal generating 
machine 101 of drawing 3 which manages the whole system generates. The 
clock of each media manages media playback according to the hour entry from 
this control section fundamentally. 

[0100] The synchronous hint information recognition section 232 receives the 
synchronous hint information acquired from the stream data decode section 21. 
[0101] The time series information collating section 233 receives the time series 
information on the time line from said time series information control section 231, 
and tests by comparison where [ on the time line ] synchronous hint information 
is applied. 

[0102] The synchronicity Monitoring Department 234 between media always 
supervises the clock of each media of the media playback section, and confirms 
whether time difference arises on the clock of media, and the clock of a system in 
response to a system-wide synchronizing signal. 

[0103] The updating part cognitive section 235 performs processing which 
secures synchronicity so that it may not be sensed, even if a gap arises between 
clocks, when it is admitted from synchronous hint information that the 
synchronicity of media playback collapses at said synchronicity Monitoring 
Department between media. 

[0104] Drawing 16 shows the flow chart explaining actuation of the 
synchronizing-hold-control section. Hereafter, with reference to this drawing, 
explanation of operation is given to drawing 15 . 

[0105] (b1) Stream data are received and data are divided into each media, 
synchronous hint information, etc. 

[0106] (b2) The contents which synchronous hint information shows are analyzed 



and a thing with sufficient performing synchronizing hold control where etc. is 
investigated in the time line which manages media synchronous playback. 
[0107] (b3) The location (time of day) which can be judged by actuation of the 
step of b2 as it is better to carry out synchronizing hold control in advance is 
recognized. 

[0108] (b4) With reference to the clock which performs system-wide time 
management, the time of day made into the location which should be carried out 
synchronizing hold control is checked in the time line at the check of the clock in 
the media playback section, and step b3. 

[0109] (b5) It investigates whether the time of day which should synchronize 
came by the check of step b4, or the gap of a synchronization arose between 
media. If it is yes, synchronizing-hold-control processing is performed, and if it is 
no, it returns to step b4. 

[01 10] (b6) It refers to with step b3, and the time of day and media which should 

be anew carried out synchronizing hold control are decided. 

[0111] (b7) In the media playback section, synchronizing hold control of the 

media decided at step b6 and time of day is performed. 

[01 12] The system which synchronizing hold control tends to perform is 

realizable by deciding beforehand the data in which synchronizing hold control is 

possible, and its contents of accommodation in media data and time of day which 

can be expected that a synchronous gap arises according to this example as 

mentioned above, and setting up the field (time of day) which can be adjusted as 

a synchronous hint. 

[0113] (Example 2) Drawing 17 is the multimedia synchronizing-hold-control 
approach according to claim 3 and the claim Fig. showing the configuration of 
multimedia synchronizing-hold-control equipment according to claim 9. 
[0114] In drawing 17 this multimedia synchronizing-hold-control approach and 
equipment A media data control means 10 to manage multimedia data and time 
line, such as an audio and video, The media data obtained from the media data 
control means 10, and a stream data generation means 11 to generate stream 



data from time-line information, It faces generating the stream data which 
perform correctly the synchronization between the media at the time of playback. 
It consists of synchronous hint information decision means 32 which acquire 
information required for synchronizing hold control from the time line which the 
media data control means 10 manages, determine a synchronous hint, and are 
given to the stream data generation means 11. 

[01 15] Drawing 18 is the multimedia synchronizing-hold-control approach 
according to claim 4 and the claim Fig. showing the configuration of multimedia 
synchronizing-hold-control equipment according to claim 10. 
[0116] In drawing 18 this multimedia synchronous playback approach and 
equipment In the configuration of the multimedia synchronous approach 
according to claim 3, or the configuration of multimedia synchronizing-hold- 
control equipment according to claim 9, in addition, a media playback means 22 
to perform a media playback output, A stream decode means 21 to decode the 
stream data generated with the stream data generation means 11 to data 
reproducible [ with the media playback means 22 ], The synchronous hint 
information given when generating stream data is acquired from the stream 
decode means 21 , and it consists of synchronizing-hold-control means 23 to 
adjust the synchronization between the media of the media playback means 22, 
referring to the time-line information which the media data control means 10 
manages. 

[0117] In the 2nd example, synchronous hint information is not given but the 
synchronous hint is determined, judging from the set-up time line. 
[01 18] Drawing 19 is the block diagram of the whole system containing the 
multimedia synchronizing-hold-control approach of this example, or equipment. 
[01 19] Although the same reference agreement is attached as the structure of a 
system since it is almost the same as the multimedia synchronizing-hold-control 
equipment of the 1st example as shown in drawing 19 , the synchronous hint 
information input section 12 which incorporates synchronous hint information 
from an external input was replaced, and it has the synchronous hint information 



decision section 32 which determines synchronous hint information from time-line 
information with reference to the time-line Management Department 102. 
[0120] Flow of operation is almost the same as the 1st example as follows. 
(1) The stream data generation section 1 1 acquires the time-line information to 
which time series information required for synchronous media playback is set 
from the time-line Management Department 102. 

[0121] (2) The stream data generation section 1 1 acquires the media data set up 
there with reference to the time line from the connection 104 with ROM 103 for 
multimedia data, a cable, and a network. 

[0122] (3) The stream data generation section 11 acquires the synchronous hint 
information for maintaining the synchronization without a gap through the 
synchronous hint information decision section 32, in case the data to create are 
reproduced. Under the present circumstances, processing as which the input of 
hint information determines a serial synchronous hint with reference to time-line 
information from the time-line Management Department 102 is performed. 
[0123] The synchronous hint information acquired at the step of (4) and (3) is 
checked with time-line information, and decision to which timing to perform 
synchronizing hold control, such as renewal of the clock of media, is performed. 
[0124] The decision performed at the step of (5) and (4) determines the data 
configuration of stream data. 

[0125] The data configuration for which it opted at the step of (6) and (5) 
generates stream data. 

[0126] Here, the actuation which determines a synchronous hint is explained 
using an example from the time line. 

[0127] In drawing 20 , TL1 and TL2 are the time line for carrying out time 
management of media playback. TL1 is what showed the time schedule by which 
each media data is reproduced, and TL2 shows independently the timing which 
loads each media data in TL1. 

[0128] Moreover, in drawing 20 , V1 , V2, and V3 express a video data, and A1 
expresses audio data. t0-t4 express specific playback time of day, and pO 



expresses the time of day when V1 and p1 start A1 , and V2 and p3 start loading 
of V3 and each data. 

[0129] Now, the time schedule of media playback is decided like the time line TL 
1 , and suppose that the time of day which each media load like the time line TL 2 
was set up mechanically. That is, in a video data V2 and the audio data A1 , time 
of day p1 to the video data v3 starts [ a video data V1 ] loading from time of day 
p3 from time of day pO, respectively. 

[0130] A very high load being applied in a video data V2 and the audio data A1, if 
loading is started by p1 to coincidence, and requiring unexpected time amount 
depending on the throughput of a system, and lapsing into the situation where a 
synchronization cannot be taken at the time of playback is also considered so 
that the start from time of day t1 may be made to meet the deadline at this time. 
[0131] Then, the synchronous hint information decision section will be considered 
that a synchronization does not shift at the time of playback if the loading point is 
early set as loading of the audio data A1 like the time of day N of drawing 
beforehand supposing taking time amount. 

[0132] Moreover, if a synchronous hint is given in addition to the example of 
drawing 20 , as a situation that synchronizing hold control can be performed 
effectively, the CPU power of a system will be used by increase of the load by 
processing of loading of data etc. across tolerance, and the case where a gap 
arises between the clocks of each media by it will be considered. 
[0133] How to give the synchronous hint in this case is explained referring to 
drawing 21 . In drawing 21 , TLo is the time line set up beforehand, and it is the 
time schedule Fig. showing that TLr has been in playback of an audio when it 
actually reproduces, and TLs is the time line to which the synchronous hint was 
set, in order to avoid the condition of TLr. 

[0134] Moreover, in drawing 21 , V4 and V5 express a video data, and A2 and A3 
express audio data. t1-t4 express specific time of day, and ts is synchronizing- 
hold-control time of day given as a synchronous hint. 

[0135] Suppose that the time schedule of media playback was now decided like 



the time line TLo. Although a video data V5 and audio data A3 will be loaded 
before time of day t3 at this time, when the data of V5 and A3 are very large, a 
situation which audio data start behind time like TLr according to the throughput 
of a system may arise. 

[0136] It makes it possible to perform synchronizing hold control at the time of 
playback by reading such a situation in the time line TLo beforehand, and 
embedding a marker with which V5 and A3 delay and adjust t3 at the refreshable 
time of day ts to coincidence without fixing playback time amount like the time 
line TLs as a synchronous hint. 

[0137] Although the two above-mentioned examples are only examples, 
according to this example, as mentioned above The synchronous hint which can 
avoid in advance the synchronous gap by the height of the load which loading of 
expectable data will take if generated at the time of playback of media etc. can 
be determined. Moreover, when a synchronous gap arises by reproducing two or 
more media, even if it adjusts the clock of which media to which timing, it is 
possible to perform synchronizing hold control in consideration of whether there 
is any trouble in playback. 

[0138] (Example 3) Drawing 22 is the multimedia synchronizing-hold-control 
approach according to claim 5 and the claim Fig. showing the configuration of 
multimedia synchronizing-hold-control equipment according to claim 11. 
[0139] In drawing 22 this multimedia synchronizing-hold-control approach and 
equipment A media data control means 10 to manage multimedia data and time 
line, such as an audio and video, The media data obtained from the media data 
control means 10, and a stream data generation means 11 to generate stream 
data from time-line information, A data-type judging means 51 to investigate the 
type of the media data which the media data control means 10 manages, It faces 
generating the stream data which perform correctly the synchronization between 
the media at the time of playback, and consists of synchronous hint information 
decision means 32 which acquire information required for synchronizing hold 
control from the type of media data, determine a synchronous hint, and are given 



to the stream data generation means 11. 

[0140] Drawing 23 is the multimedia synchronizing-hold-control approach 
according to claim 6 and the claim Fig. showing the configuration of multimedia 
synchronizing-hold-control equipment according to claim 12. 
[0141] In drawing 23 this multimedia synchronous playback approach and 
equipment In the configuration of the multimedia synchronous approach 
according to claim 6, or the configuration of multimedia synchronizing-hold- 
control equipment according to claim 12, in addition, a media playback means 22 
to perform a media playback output, A stream decode means 21 to decode the 
stream data generated with the stream data generation means 1 1 to data 
reproducible [ with the media playback means 22 ], The synchronous hint 
information given when generating stream data is acquired from the stream 
decode means 21, and it consists of synchronizing-hold-control means 23 to 
adjust the synchronization between the media of the media playback means 22, 
referring to the time-line information which the media data control means 10 
manages. 

[0142] In the 3rd example, a synchronous hint is not determined, judging from the 
set-up time line, but information, such as a type of the media data used and 
capacity, is investigated, and a synchronous hint is determined. 
[0143] A data-type judging means to give the information on a data type etc. to 
the synchronous hint information decision section is only added, and the 
configuration is carrying out the almost same configuration as the 2nd example. 
[0144] Flow of operation is almost the same as the 2nd example. Here, the 
actuation which determines a synchronous hint is explained using an example 
from the type of data. 

[0145] (a) of drawing 24 shows briefly the data information from which the size of 
the data to be used etc. was set as the media data control means in a format like 
the structure. It is possible that actual more complicated and various information 
is set up. 

[0146] A data-type judging means reads this information, and a video data V7 



can be judged that the data playback section must carry out loading of the data 
from quite early time amount in this example from balance with the video data by 
which a synchronous hint information decision means has a large capacity, 
judging from this information, and is reproduced in front. 
[0147] The example which set up the synchronous hint from the type of data in 
advance by this is (b) of drawing 24 . 

[0148] In this drawing, TLa is what showed the time schedule by which each 
media data is reproduced, and TLb shows independently the timing which loads 
each media data in TLa. 

[0149] Moreover, in this drawing, V6 and V7 express a video data, and A4 
expresses audio data. t0-t2 express specific playback time of day, and the time 
of day when pO starts V6 and p1 starts loading of the data of V7 and each A4 is 
expressed. 

[0150] Now, the time schedule of media playback is decided like the time line 
TLa, and suppose that the time of day which each media load like the time line 
TLb was set up mechanically. That is, in a video data V6, a video data V7 and 
audio data A4 start loading from time of day p1 from time of day pO, respectively. 
[0151] According to the contents judged from the data information of (a) at this 
time, a very high load being applied if the video data V7 with a, big capacity is 
loaded during playback of a video data V6, and requiring unexpected time 
amount depending on the throughput of a system, and lapsing into the situation 
where a synchronization cannot be taken at the time of playback is also 
considered. 

[0152] Then, the hint information to which it is made for loading of V7 to begin 
from early like the loading point V of TLb as timing which loads V7 is given. 
[0153] Moreover, since audio data A4 repeats the same data with small size and 
is only reproduced, loading is started from the time of day A later than p1 , i.e., 
the loading point of TLb, and the hint information which is made not to interfere 
with loading of a video data is given. 

[0154] Thus, the media playback which a synchronous gap does not produce is 



attained by investigating the information on data in advance and giving hint 

information. 

[0155] 

[Effect of the Invention] As explained above, in order to avoid the gap which may 
be produced in case two or more media are reproduced to coincidence according 
to the multimedia synchronizing-hold-control approach concerning invention of 
claim 1, and the multimedia synchronizing-hold-control equipment concerning 
invention of claim 7, the guarantee of the synchronicity of multimedia playback 
can be offered easy how synchronizing hold control should be performed in the 
phase which generates stream data, and by giving a hint. 

[0156] Moreover, in order to avoid the gap which may be produced in case two or 
more media are reproduced to coincidence, a synchronous gap of multimedia 
playback is avoidable according to the multimedia synchronizing-hold-control 
approach concerning invention of claim 3, and the multimedia synchronizing- 
hold-control equipment concerning invention of claim 9, in advance by 
determining the synchronous hint information given beforehand with reference to 
the time line in the phase which generates stream data. 

[0157] Moreover, in order to avoid the gap which may be produced in case two or 
more media are reproduced to coincidence, a synchronous gap of multimedia 
playback is avoidable according to the multimedia synchronizing-hold-control 
approach concerning invention of claim 5, and the multimedia synchronizing- 
hold-control equipment concerning invention of claim 11 in advance by 
determining the synchronous hint information given beforehand with reference to 
the type of the media data to be used in the phase which generates stream data. 
[0158] Moreover, according to the multimedia synchronizing-hold-control 
approach concerning claims 2, 4, and 6, and the multimedia synchronizing-hold- 
control equipment concerning invention of claims 8, 10, and 12, the guarantee of 
the synchronicity of multimedia playback can be offered easy by performing 
synchronizing hold control between the media reproduced based on the 
synchronous hint information that the stream with synchronous hint information 



was decoded and obtained. 
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[Brief Description of the Drawings] 

[Drawing 1] Claim 1 and the block diagram corresponding to a claim showing the 
basic configuration of invention given in seven 

[Drawing 2] Claim 2 and the block diagram corresponding to a claim showing the 
basic configuration of invention given in eight 
[Drawing 3] The structure-of-a-system Fig. where it has multimedia 
synchronizing-hold-control equipment of the 1st example 
[Drawing 4] Drawing showing an example of a time-line interface 
[Drawing 5] The flow chart of the stream data origination actuation with the 
synchronous hint information in the multimedia synchronizing-hold-control 
equipment of the 1st example 

[Drawing 6] Drawing showing an example of the synchronous hint given to data 

[Drawing 7] The flow chart which shows the example of the processing process 

of the media synchronousr control by the synchronous hint 

[Drawing 8] The flow chart which shows the example of the processing process 

of the media synchronousr control by the synchronous hint 

[Drawing 9] The flow chart which shows the example of the processing process 

of the media synchronousr control by the synchronous hint 

[Drawing 10] The flow chart which shows the example of the processing process 

of the media synchronousr control by the synchronous hint 

[Drawing 1 1] Drawing showing an example of the media synchronousr control by 

the synchronous hint 

[Drawing 12] Drawing showing an example of the media synchronousr control by 
the synchronous hint 



[Drawing 13] Drawing showing an example of the media synchronousr control by 
the synchronous hint 

[Drawing 14] Drawing showing an example of the media synchronousr control by 
the synchronous hint 

[Drawing 15] The internal configuration Fig. of the stream data decode section 21 , 
the media playback section 22, and the synchronizing-hold-control section 23 
[Drawing 16] The flow chart of actuation of adjusting a synchronous gap from 
synchronous hint information 

[Drawing 17] Claim 3 and the block diagram corresponding to a claim showing 
the basic configuration of invention given in nine 

[Drawing 18] Claim 4 and the block diagram corresponding to a claim showing 

the basic configuration of invention given in ten 

[Drawing 19] The structure-of-a-system Fig. where it has multimedia 

synchronizing-hold-control equipment of the 2nd example 

[Drawing 20] Drawing showing an example of actuation which gives a 

synchronous hint according to the time line (1) 

[Drawing 21] Drawing showing an example of actuation which gives a 
synchronous hint according to the time line (2) 

[Drawing 22] Claim 5, the block diagram corresponding to a claim showing the 
basic configuration of invention of 1 1 publications 

[Drawing 23] Claim 6, the block diagram corresponding to a claim showing the 

basic configuration of invention of 12 publications 

[Drawing 24] Drawing showing an example of actuation which gives a 

synchronous hint according to the information on data 

[Drawing 25] The time sequence diagram having shown the example of the 

synchronous gap explaining Object of the Invention 

[Description of Notations] 

10 Media Data Control Means 

1 1 Stream Data Generation Means and Stream Data Generation Section 

12 Synchronous Hint Information Input Means and Synchronous Hint Information 



Input Section 

21 Stream Decode Means and Stream Data Decode Section 

22 Media Playback Means and Media Playback Section 

23 Synchronizing-Hold-Control Means and Synchronizing-Hold-Control Section 
32 Synchronous Hint Information Decision Means and Synchronous Hint 
Information Decision Section 

51 Data-Type Judging Means 

100 CPU 

101 Synchronizing Signal Generating Machine 

102 Time-Line Management Department 

103 ROM for Multimedia Data 

104 Cable, Connection with Network 

105 User Interface 

106 Working-level Month RAM 

211 Stream Data Decomposition Section 

212 Synchronous Hint Information Output Section 

213 Time Series Information Output Section 

214 Audio Data Output Section 

215 Video-Data Output Section 

221 Audio Control Section 

222 Video Control Section 

223 Audio Clock 

224 Video Recovery Clock 

225 DRAM 

226 VRAM 

227 Audio Device 

228 Display 

231 Time Series Information Control Section 

232 Synchronous Hint Information Recognition Section 

233 Time Series Information Collating Section 



234 Synchronicity Monitoring Department between Media 

235 Updating Part Cognitive Section 

236 Renewal Section of Synchronicity 



